Search suggestion and display environment

ABSTRACT

Methods, systems, and apparatus for processing image content items. In one aspect, a method includes receiving a set of search query suggestions for a query, each search query suggestion being based on the search query; for each search query suggestion, receiving data identifying a respective set of resources determined to be responsive to the search query suggestion; for each respective set of resources, generating a representative resource set depicting a subset of the respective set of resources; providing to the user device, in response the search query, a search results page resource including instructions that cause the user device to render a search results page that includes: a set of search results that identify resources that are determined to be responsive to the search query; for each search query suggestion, a search query suggestion grouping including: data describing the search query suggestion, and data depicting the representative resource set.

BACKGROUND

This specification relates to information presentation.

The Internet provides access to a wide variety of resources. Forexample, digital image files, video and/or audio files, as well aswebpages for particular subjects or particular news articles, areaccessible over the Internet. Each of these digital files and webpagesare generally referred to as a resource. A search engine can identifyresources in response to a user query that includes one or more searchterms or phrases. The search engine ranks the resources based on theirrelevance to the query and importance and provides search results thatlink to the identified resources, and orders the search resultsaccording to the rank.

For example, digital images that satisfy a user's informational need canbe identified by an image search process in which keywords or other dataare processed to identify collections of digital images. Each image isidentified to the user by an image search result. An image search resultreferences a corresponding image resource that a search enginedetermines to be responsive to a search query, and typically includes arepresentative image of the image resource, such as a thumbnail. Theimage search result also references a webpage resource that includes theimage resource.

SUMMARY

In general, one innovative aspect of the subject matter described inthis specification can be embodied in methods that include the actionsof receiving, in response to a search query for resources received froma user device, a set of search query suggestions, each search querysuggestion being based on the search query and different from the searchquery; for each search query suggestion, receiving data identifying arespective set of resources determined to be responsive to the searchquery suggestion; for each respective set of resources, generating arepresentative resource set depicting a subset of the respective set ofresources; providing to the user device, in response the search query, asearch results page resource, the search results page resource includinginstructions that cause the user device to render a search results pagethat includes: a set of search results that identify resources that aredetermined to be responsive to the search query, for each search querysuggestion, a search query suggestion grouping including: datadescribing the search query suggestion and data depicting therepresentative resource set. Other embodiments of this aspect includecorresponding systems, apparatus, and computer programs, configured toperform the actions of the methods, encoded on computer storage devices.

Another innovative aspect of the subject matter described in thisspecification can be embodied in methods that include the actions ofreceiving, in response to a search query for resources received from auser device, a set of search query suggestions, each search querysuggestion being based on the search query and different from the searchquery; for each search query suggestion, receiving data identifying arespective set of resources determined to be responsive to the searchquery suggestion; for each respective set of resources, generating arepresentative resource set depicting a subset of the respective set ofresources; providing to the user device, in response the search query, asearch results page resource, the search results page resource includinginstructions that cause the user device to render a search results pagethat includes: a set of search results that identify resources that aredetermined to be responsive to the search query, for each search querysuggestion, a search query suggestion grouping including: datadescribing the search query suggestion, and data depicting therepresentative resource set. Other embodiments of this aspect includecorresponding systems, apparatus, and computer programs, configured toperform the actions of the methods, encoded on computer storage devices.For a system of one or more computers to be configured to performparticular operations or actions means that the system has installed onit software, firmware, hardware, or a combination of them that inoperation cause the system to perform the operations or actions. For oneor more computer programs to be configured to perform particularoperations or actions means that the one or more programs includeinstructions that, when executed by data processing apparatus, cause theapparatus to perform the actions.

Particular embodiments of the subject matter described in thisspecification can be implemented so as to realize one or more of thefollowing advantages. Displaying a query suggestion grouping for eachquery suggestion provides users with a representative resource contextfor the query suggestion, which facilitates a visual analysis of thecontext of the query suggestion. Such visual analysis of the context canhelp users discern whether the search suggestion will help satisfyinformation needs.

Additionally, a user can quickly review a number of search results for aquery suggestion in the context of other search results for the query todecide which set of search results best satisfies the user'sinformational need. This facilitates a contextual analysis of searchresult sets by the user, and eliminates the need for the user to openmultiple display tabs or active windows to review the search results.

The details of one or more embodiments of the subject matter describedin this specification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example environment in which imagesearch results are processed.

FIGS. 2A-2C are illustrations of example user interfaces presentedduring the processing of a query suggestion grouping.

FIG. 3 is a block diagram illustrating the selection of a representativeresource set.

FIG. 4 is a flowchart of an example process for generating and providingquery suggestion groupings.

FIG. 5 is an illustration of query suggestion groupings that include awebsite grouping.

FIG. 6 is a flowchart of an example process for processing querysuggestion groupings at a user device.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of an example environment 100 in which systemsimplementing the subject matter of this written description areimplemented. A network 102, such as a local area network (LAN), a widearea network (WAN), the Internet, or a combination thereof, connectswebsites 104, user devices 106, and a search system 112.

A website 104 includes one or more resources 105 associated with adomain name and hosted by one or more servers. An example website is acollection of webpages formatted in hypertext markup language (HTML)that can contain text, images, multimedia content, and programmingelements, such as scripts. Each website 104 is maintained by a contentpublisher, which is an entity that controls, manages and/or owns thewebsite 104.

A resource 105 can be any data that can be provided over the network102. A resource 105 can be identified by a resource address that isassociated with the resource 105. Resources include HTML pages, images,portable document format (PDF) documents, videos, and feed sources, toname only a few. Additionally, resources such as webpage resources can,in turn, reference other resources such as images so that when thewebpage resource is rendered on a user device, the referenced image isalso displayed.

A user device 106 is an electronic device that is capable of requestingand receiving resources over the network 102. Example user devices 106include personal computers, mobile communication devices, e.g.,smartphones, and other devices that can send and receive data over thenetwork 102. A user device 106 typically includes one or more userapplications, such as a web browser, to facilitate the sending andreceiving of data over the network 102. By use of these applications, auser device 106 can request resources 105 from a website 104. In turn,the resource 105 can be provided to the user device 106 for presentationby the user device 106.

To facilitate searching of these resources 105, the search engine 110identifies the resources by crawling the publisher websites 104 andindexing the resources 105 provided by the publisher websites 104. Theresources 105 are indexed and the index data are stored in an index 112.

The user devices 106 submit search queries to the search engine 110. Inresponse to the search request, the search engine 110 uses the index 112to identify resources that are relevant to the queries. The searchengine 110 identifies the resources in the form of search results andreturns the search results to the user devices 106 in a search resultspage resource 111. A search result is data generated by the searchengine 110 that identifies a resource or provides information thatsatisfies a particular search query. A search result for a resource caninclude a webpage title, a snippet of text extracted from the webpage,and a resource locator for the resource, e.g., the URL of a webpage. Inthe case of images, the search result may include a representative image(e.g., a thumbnail) of the image and the resource locator for the image.For example, the search system 110 utilizes image processing algorithmsto identify multiple instances of the same image. The search system 110,in some implementations, then selects a representative image, e.g., acanonical image, that is used to represent each of the identical images,and associates the representative image with each of the underlyingidentical images. The underlying webpage that is referenced in the imagesearch result is determined at query time, as a particular query mayinclude information that results in one particular webpage beingselected over other webpages that include the same image. The searchresults are ranked based on scores related to the resources identifiedby the search results, ordered according to these scores, and providedto the user device according to the order.

The search system 110 returns the search result to the user devices 106in the form of a search results page. The search result page typicallyincludes multiple search results. The user device 106 receives thesearch results page and renders the page for presentation.

In some implementations, the search queries 109 submitted from userdevices 106 are stored in query logs 114. Selection data for the queriesand the webpages referenced by the search results and selected by usersare stored in selection logs 116. The query logs 114 and the selectionlogs 116 define search history data 117 that include data from andrelated to previous search requests associated with unique identifiers.The selection logs represent actions taken responsive to search resultsprovided by the search engine 110. The query logs 114 and selection logs116 can be used to map queries to resources that were identified insearch results and the actions taken by users when presented with thesearch results in response to the queries. The selection logs 116 andquery logs 114 can thus be used by the search engine 110 to determinethe respective sequences of queries submitted by the user devices 106,the actions taken in response to the queries, and to generatesuggestions to submitted queries.

Often a user may submit more than one query during a search session.This is because a user may often refine a query while searching forinformation. The search engine 110 includes a suggestion generator toassist users in refining queries. For a particular query input by auser, the search engine 110 may provide one or more suggestions for thesearch results. A user may select one of the query suggestions toreceive search results responsive to the query suggestion.

In some implementations, the search engine 110 includes a querysuggestion grouping generator 122 that generates a query suggestiongrouping for each query suggestion. Each query suggestion groupingincludes data describing the query suggestion and data depicting arepresentative resource set of resources that are responsive to thequery. Illustrations of such groupings for search results are describedin more detail below in the context of image search results. However,the groupings can be used for any type of resource search operation. Forexample, query suggestion groupings can be generated for searchsuggestions for webpage resource searches, video searches, productsearches, book searches, and so on. Thus, the features described beloware not limited to image search suggestions.

As shown in FIG. 1, the user device 106 submits a query to the searchengine 110 after the user inputs the query on a search interface 130. Inresponse to the query, the search engine 110 returns a search resultspage 134 that includes query suggestion groupings 136 and search results138. In some implementations, that query suggestion groupings 136 arearranged in a row above search results 138. The search results 138 canbe provided in the form of a list, such as in the case of search resultsfor webpages, or, as shown in FIG. 1, in the form of rows, such as inthe case of images or videos. Other arrangements of query suggestiongroupings can also be used, however.

A more detailed diagram of an example search results page with querysuggestions is shown in FIG. 2A. In FIG. 2A, the query 204 [rhubarb] isinput for a search of an image corpus. The query suggestion groupings210, 212, 214 and 216 are arranged in a row above the rows of imagesearch results SR11-1-SR35-1. Each image search result references acorresponding image resource that is determined to be responsive to asearch query and includes a representative image of the image resource.For example, each image shown in the search results of the searchresults page 200 may be a smaller version of a corresponding imageprovided by various websites 104. Each search result references a pageresource 105 that includes the image resource. The page resource may, insome situations, be the same as the image resource; alternatively, thepage resource may be an HTML page that references the image resource sothat when the HTML page is rendered on a user device, the image isdisplayed with additional content.

Each query suggestion grouping in FIG. 2A includes text describing thequery suggestion and image data depicting a representative image set ofimages. Each image in the representative image set is responsive to thequery suggestion. For example, as shown in FIG. 2A, for the query[rhubarb], query suggestion groupings for the query suggestions [rhubarbstalk], [rhubarb plant], [rhubarb leaves] and [rhubarb garden] areshown. In some implementations, the textual data describing the querysuggestion is non-duplicative of text of the search query. For example,as shown in FIG. 2A, for the query [rhubarb], only the text “stalk,”“plant,” “leaves” and “garden” are shown for the query suggestions[rhubarb stalk], [rhubarb plant], [rhubarb leaves] and [rhubarb garden],respectively.

The query suggestion grouping generator can determine non-duplicativetext in a variety of ways. For example, one way is to eliminate allterms that appear in the original query. Another way is to identifydistinct semantic entities, such as n-grams that have an associatedinformation gain, within the query suggestion, and provide the n-gram asthe textual data describing the query suggestion. In the case of thelatter, one or more terms in the original query may appear in thetextual data describing the query suggestion, but only those terms thatare necessary to specify the distinct semantic entity are included.Thus, in the latter case, non-duplicative text does not necessarilyrequire mutually exclusive terms. For example, consider the querydirected to a famous person, e.g., [Ima Starr now]. One query suggestionmay be [Ima Starr now and then]. Because the tri-gram “now and then” isa distinct semantic entity, the text describing the query refinement[Ima Starr now and then] is “now and then” and not “and then.”

For image search results, the data depicting the image set can includethumbnails of responsive images. The thumbnails are arranged in variousgeometric configurations as shown in FIG. 2A. The configurations mayvary based on, for example, the aspect ratio of the images and theranking of the images in response to each query suggestion, and thenumber of thumbnail representations that can fit in an allotted area Awithout undergoing a maximum scaling reduction Z. For example, for thequery suggestion [rhubarb stalk], the top three ranked images may be afirst image having a square aspect ratio, and second and third imagesthat each have a 2×4 aspect ratio. Likewise, for the query suggestion[rhubarb plant], the top four ranked images may be first and secondimages that each have a 2×4 aspect ratio, and third and fourth imagesthat each have a 4×2 aspect ratio. For the first grouping 210, thethumbnails are of such aspect ratios that only three thumbnails fit intothe allotted area A without exceeding the scaling reduction Z. However,for the second grouping 212, the thumbnails are of such aspect ratiosthat four thumbnails fit into the allotted area A without exceeding thescaling reduction Z.

For the query grouping 210, the thumbnails may be of images of rhubarbstalks; for the grouping 212, the thumbnails may be of rhubarb plants;and so on. The representative image set may be a single thumbnail imagegenerated from respective images, or, alternatively, may be a set ofseparate thumbnails that are rendered in an adjacently joined set toform a single grouping.

In some implementations, the representative image set depicting a subsetof a respective set of images determined to be responsive to the searchquery suggestion is selected such that none of the images in therepresentative image is an image depicted in the search resultsSR11-1-SR35-1 shown on the search results page. The selection of therepresentative image set is described with reference to FIG. 3 below.

A user may select a query suggestion grouping to evaluate the querysuggestion. As shown in FIG. 1, the user has selected the second querysuggestion grouping that includes the query suggestion textualdescription QST2. In response to a selection of a particular querysuggestion grouping, the user device executes instructions within thesearch results page 134 that cause the user device to generate a querysuggestion display environment 140 on the search results page 134. Forexample, the query suggestion display environment 140 is generatedbetween the first row in which the query suggestions groupings 136 areshown and the first row of the image search results 138. As a result,the image search results 138 are pushed downward, and some of the searchresults 138 may move out of the view port, as shown.

The query suggestion display environment displays at least the subset142 of the respective set of images determined to be responsive to thequery suggestion for which the query suggestion grouping is selected,and at least the textual data describing the query suggestion. Forexample, as shown in FIG. 1, the environment 140 includes six thumbnailsand the entire query suggestion QR2.

FIG. 2B is a more detailed illustration of the search results pagedisplaying the query suggestion display environment. In FIG. 2B, theuser has selected the query suggestion grouping 212. The selection canbe, for example, the result of a hover in excess of a hover time, aclick on the query suggestion grouping 212, or some other signal of userselection or interest. In response to the selection, the querysuggestion display environment 220 is generated adjacent to the querysuggestion groupings. An indicator 228 on the display environment 220border indicates the query suggestion grouping the first row to whichthe image display environment 220 corresponds.

The query suggestion display environment 220 includes search results 222and the query suggestion 224. In some implementations, the searchresults 222 that are shown are the top N search results for the querysuggestion 224, and include images for search results that were depictedin the query suggestion grouping for which the query suggestionenvironment 220 was generated. For example, as shown in FIG. 2B, theimages for the search results SR11-2-SR14-2 are represented bythumbnails in the query suggestion grouping 212. In someimplementations, selection of one of the search results 222 causes theuser device to request the image depicted in the search result, or,alternatively, request the webpage resource that includes the imagedepicted in the search result. Selecting the “View more images” button,or clicking on the query suggestion grouping 212 when the displayenvironment 220 is open, causes the user device to submit the querysuggestion to the search engine as a query. For example, assume the useris interested in the search results shown in the environment 220, anddesires to see additional search results. The user can select either“View more images” button and a new search results page will begenerated, such as the search results page 150 shown FIG. 1. The searchresults page 150 includes the query suggestion QR2 as the input query,and the search results 158 are responsive to the query suggestion.Furthermore, additional query groupings 156 are generated for the queryQR2.

FIG. 2C is a more detailed illustration of a search results page 240resulting from a selection of a query suggestion grouping. The searchresults page 240 is similar to the search results page 200, except thatthe query suggestion groupings 250, 252, 256 and 258 differ from thequery suggestion groupings 210, 212, 214 and 216, as do the underlyingsearch results.

FIG. 3 is a block diagram 300 illustrating the selection of arepresentative content item set. The process attendant to the blockdiagram is described with respect to FIG. 4. In some implementations,the process 400 of FIG. 4 is run at query time each time a query isreceived. In other implementations, portions of the process may be runperiodically for a set of most popular queries received by the searchengine. The run time selected may be a time during which system resourcerequirements are relatively low, e.g., during early morning hours. Theresulting query suggestion groupings are thus predetermined andpre-cached for various popular queries, and when a query is received atquery time, the corresponding pre-cached query suggestion groupings canbe fetched from memory. The frequency at which the query suggestiongroupings are determined can be based on one of various predefinedperiods, e.g., daily, weekly, etc.

For a particular query 310 input by a user, the query suggestiongrouping generator 122 receives data identifying resources 312responsive to the query (402). Examples of such data can be the URL ofthe resource, or a search result that includes the URL of the resourceand data from the resource, such as a snippet of text or a thumbnail, orother data that can be used to identify the resource.

For the search query, the query suggestion grouping generator 122receives a set of search query suggestions for the search query (404).Each of the search query suggestion 312 is based on the search query 310and is different from the query 310. A variety of appropriate searchquery suggestion processes can be used to generate search querysuggestions. In FIG. 3, one query suggestion 320 is shown to avoidcongestion in the drawing.

For each search query suggestion 320, the query suggestion groupinggenerator 122 receives data identifying a respective set of resources322 determined to be responsive to the search query suggestion (406).Again, examples of such data can be the URL of the resource, or a searchresult that includes the URL of the resource and data from the resource,such as a snippet of text or a thumbnail, or other data that can be usedto identify the resource. The data identifying the resources alsospecifies a ranking of the resources in response to the query, whereeach resource is ranked according to an ordinal position relative toother resources that are responsive to the query.

For each respective set of responsive resources 322, the querysuggestion grouping generator 122 generates a representative resourceset 330 depicting a subset of the respective set of resources (408). Thequery suggestion grouping also includes textual data 332 as describedabove.

In some implementations, the query suggestion grouping generator 122selects the top N ranked resources. The value of N may be predefined,e.g., 3, 4, or 5, for example, or may vary depending on the aspectratios that are required to depict each resource and the amount of areaavailable in which the representative resource set can be depicted.

In some implementations, the resources that are selected to generate therepresentative set can be selected from the respective resources thatare not identified in the top M search results for the query, where M isgreater than N. For example, M may be 20, 30 or 40, or some other value.As shown in FIG. 3, the responsive resources 322 that are shaded areresources that are also within the top M resources 312 for the query310. Accordingly, these resources are not used to generate therepresentative set 330 for the query grouping, as indicated by the “x”detail respectively descending from each of the resources.

In some implementations, a query suggestion grouping for a querysuggestion is only generated if the resources 322 for the querysuggestion do not meet an overlap threshold with the resources 312 forthe query 310 for the top M resources in both sets 312 and 322. Forexample, assume that the overlap threshold is a predefined percentage.If more than a predefined percentage of the responsive resources 322overlap with the resources 312 within the top N resources, then a querygrouping will not be generated for the query suggestion, nor will thequery suggestion be provided in response to the query.

In some implementations, the resources that are selected to generate therepresentative set can be selected from any of the respective resourcesas long as the overlap threshold is not met. Other appropriate ways ofselecting resources can also be used.

In response to the query, the search engine 110 provides a searchresults resource that includes a set of search results that identifyresources that are determined to be responsive to the query, and, foreach query suggestion, a query suggestion grouping that includes datadescribing the query suggestion and data depicting the representativeresource set (410). For example, a search results page such as thesearch results page 200 of FIG. 2A is provided to a user device fromwhich the query was received.

While the query suggestion groupings described above are for aparticular query, e.g., a most recent query, the query suggestiongroupings can be based on several or more previous queries for aparticular user device or user session. For example, the querysuggestion grouping include one grouping for a query suggestion for amost recently input query, and one or more groupings for one or morequeries input prior to the most recently input query.

While the query suggestion groupings described above are for querysuggestions and correspondingly responsive search results, other typesof query suggestion groupings can also be generated. For example, thequery suggestion grouping may include a link to a website that isdifferent from the search engine and that includes a collection ofresources that may be of interest to the user. Thus, the querysuggestion may be descriptive of a content collection, and therepresentative resource set may be a subset of a collection of resourcesthat belong to the content collection.

FIG. 4 is an illustration of query suggestion groupings that include awebsite grouping as one of the query suggestion groupings. For the query[Geometric Art], two query suggestion groupings 500 and 510 as describedabove are generated by the query suggestion grouping generator 122.However, a third query suggestion grouping 520 that includes a link to awebsite that is different from the search engine is generated. Forexample, the artist “John Doe” may be an artist that creates geometricshaped art, and there may be a particular website that features imagesof his collection. Thus, in some implementations, the query suggestiongrouping generator 122 generates a grouping that is descriptive of thecontent of the collection available at the website. Selection of thequery suggestion grouping will cause the display environment to displayresources from the collection. The display environment may also includea link to the web site at which the collection is hosted.

The query suggestion grouping generator 122 can utilize a variety ofappropriate processes can be used to determine whether a querysuggestion grouping should reference responsive resources form a hostedcollection at a particular website or domain rather than individualresources from different sites. For example, for a particular querysuggestion, e.g., [Geometric Art John Doe], the query suggestiongrouping generator 122 determines a diversity measure that measures thedomain diversity of respective domains on which the top K resourcesresponsive to the query are hosted. For example, the domain diversityscore may be a ratio of the number of resources J in the top K resourceshosted on a host that belongs to the particular domain, e.g., J/K. Ifthe ratio exceeds a percentage, then the query suggestion groupinggenerator 122 generates a grouping that is descriptive of the content ofthe collection available at the website belonging to the domain.

In another implementation, a page diversity score can be used. The pagediversity score is similar to the domain diversity score, and is a ratioof the number of resources J in the top K resources hosted on aparticular webpage. For example, the “John Doe Collection 2103” maydepicted by a set of images that are displayed on a single webpage.

To calculate the diversity score, query suggestion grouping generator122 identifies a common address unit for each resource and groups theresources by the common address units. A common address unit may be, forexample, a domain name, a complete URL, etc. Furthermore, multiplediversity checks for different common address unit types can be donesimultaneously. For each common address unit that has a diversity scorethat meets a threshold, a query suggestion grouping can be generated.

Additional information can also be included in the query suggestiongrouping. For example, as shown in FIG. 5, instead of the querysuggestion [Geometric Art John Doe], the description [John DoeCollection 2013] is shown. The description can be, for example, takenfrom a webpage title, derived from links to the webpage or site, or fromsome other source. Likewise, the data depicting the representativeresource set also includes an image 522 of the author John Doe. Theimage may be selected based on either its relative importance andresponsiveness to the query suggestion [Geometric Art John Doe] orrelative importance in the collection of images within the webpage orweb site at which the images are located.

In another implementation, the query suggestion grouping can be acollection of images from a particular web page, and the web page isselected when a search score (or ranking) of the web page meets aminimum scoring (or ranking) threshold.

In yet another implementation, the query suggestion grouping candescribe a particular feature of a collection of images that areresponsive to the query or the query suggestion. For example, aparticular set of images relating to the same topic (e.g., the EiffelTower), may be processed to identify distinct visual features by whichthe set of images may be partitioned. For example, the distinct visualfeatures may be black and with images, sepia tone images, images takenat night, images taken in full sunlight, etc. Each query suggestiongrouping may thus include a visual and textual description of the visualfeatures, e.g., images in black and white may include black and whitethumbnails and include the text “Black and White”, and so on. Imagemetadata may also be processed to partition a set of images. Forexample, images may be portioned by photographic date ranges, by imagesize, by location from which the photographs were taken, etc.

In still other implementations, query suggestion groupings can includedata from entity relations of a knowledge graph. For example, an entitymay be a particular author, and the knowledge graph may include a listof books authored by the author. Accordingly, the query suggestiongrouping may include an image of the author, images of books authored bythe author, and text describing the content of the grouping, e.g.,“Books by Jane Doe.”

FIG. 6 is a flowchart of an example process 600 for processing querysuggestion groupings at a user device. The operations are caused by theuser device executing instructions included in the search results pageresource.

The user device renders the query suggestion groupings in a first row onthe search results page (602). For example, as shown in FIG. 2, thequery suggestion groupings can be rendered above the search results andbelow the query input field. Alternatively, the query suggestiongroupings can be rendered vertically on a side of the search resultspage, or at the bottom of the search results page, or in between a setof rows of search results.

The user device renders the set of search results in rows adjacent thefirst row on the search results page (604). The row can include columns,such as in the case of image search results, or can be single elements,such as in the case of webpage (textual) search results.

The user device monitors for a selection of a query suggestion grouping(606). For example, the user device monitors for a mouse over or hoverover a search query suggestion grouping.

In response to a selection of a search query suggestion grouping, theuser device generates a query suggestion display environment on thesearch results page between the first row and the first row of thesearch results (608). For example, a display environment as depicted inFIG. 2B is generated.

The user device displays, in the display environment, at least thesubset of the respective set of resources determined to be responsive tothe query suggestion for which the query suggestion grouping is selected(610). For example, representations of the resources (or the actualresources, depending on the type of resource being searched) may bedisplayed as search results 222 as depicted in FIG. 2B.

The user device displays textual data describing the query suggestion(612). For example, the entire search query suggestion may be displayedas shown in FIG. 2B.

Embodiments of the subject matter and the operations described in thisspecification can be implemented in digital electronic circuitry, or incomputer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Embodiments of the subject matterdescribed in this specification can be implemented as one or morecomputer programs, e.g., one or more modules of computer programinstructions, encoded on computer storage medium for execution by, or tocontrol the operation of, data processing apparatus. Alternatively or inaddition, the program instructions can be encoded on anartificially-generated propagated signal, e.g., a machine-generatedelectrical, optical, or electromagnetic signal that is generated toencode information for transmission to suitable receiver apparatus forexecution by a data processing apparatus. A computer storage medium canbe, or be included in, a computer-readable storage device, acomputer-readable storage substrate, a random or serial access memoryarray or device, or a combination of one or more of them. Moreover,while a computer storage medium is not a propagated signal, a computerstorage medium can be a source or destination of computer programinstructions encoded in an artificially-generated propagated signal. Thecomputer storage medium can also be, or be included in, one or moreseparate physical components or media (e.g., multiple CDs, disks, orother storage devices).

The operations described in this specification can be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources.

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, a system on a chip, or multipleones, or combinations, of the foregoing The apparatus can also include,in addition to hardware, code that creates an execution environment forthe computer program in question, e.g., code that constitutes processorfirmware, a protocol stack, a database management system, an operatingsystem, a cross-platform runtime environment, a virtual machine, or acombination of one or more of them. The apparatus and executionenvironment can realize various different computing modelinfrastructures, such as web services, distributed computing and gridcomputing infrastructures.

A computer program, also known as a program, software, softwareapplication, script, or code, can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data, e.g., one or more scripts stored in a markup language document,in a single file dedicated to the program in question, or in multiplecoordinated files, e.g., files that store one or more modules,sub-programs, or portions of code. A computer program can be deployed tobe executed on one computer or on multiple computers that are located atone site or distributed across multiple sites and interconnected by acommunication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data, e.g., magnetic, magneto-optical disks, or optical disks.However, a computer need not have such devices. Moreover, a computer canbe embedded in another device, e.g., a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storage device(e.g., a universal serial bus (USB) flash drive), to name just a few.Devices suitable for storing computer program instructions and datainclude all forms of non-volatile memory, media and memory devices,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a computerhaving a display device, e.g., a CRT (cathode ray tube) or LCD (liquidcrystal display) monitor, for displaying information to the user and akeyboard and a pointing device, e.g., a mouse or a trackball, by whichthe user can provide input to the computer. Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input. In addition, a computer can interact with a user bysending documents to and receiving documents from a device that is usedby the user; for example, by sending webpages to a web browser on auser's user device in response to requests received from the webbrowser.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back-end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front-end component, e.g., aclient computer having a graphical user interface or a Web browserthrough which a user can interact with an implementation of the subjectmatter described in this specification, or any combination of one ormore such back-end, middleware, or front-end components. The componentsof the system can be interconnected by any form or medium of digitaldata communication, e.g., a communication network. Examples ofcommunication networks include a local area network (“LAN”) and a widearea network (“WAN”), an inter-network (e.g., the Internet), andpeer-to-peer networks (e.g., ad hoc peer-to-peer networks).

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someembodiments, a server transmits data, e.g., an HTML page, to a userdevice, e.g., for purposes of displaying data to and receiving userinput from a user interacting with the user device. Data generated atthe user device, e.g., a result of the user interaction can be receivedfrom the user device at the server.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular embodiments of particular inventions.Certain features that are described in this specification in the contextof separate embodiments can also be implemented in combination in asingle embodiment. Conversely, various features that are described inthe context of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

Thus, particular embodiments of the subject matter have been described.Other embodiments are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous.

What is claimed is:
 1. A method performed by a data processingapparatus, the method comprising: receiving, in response to a searchquery for resources received from a user device, a set of search querysuggestions, each search query suggestion being based on the searchquery and different from the search query; for each search querysuggestion, receiving data identifying a respective set of resourcesdetermined to be responsive to the search query suggestion; for eachrespective set of resources, generating a representative resource setdepicting a subset of the respective set of resources; providing to theuser device, in response the search query, a search results pageresource, the search results page resource including instructions thatcause the user device to render a search results page that includes: aset of search results that identify resources that are determined to beresponsive to the search query; for each search query suggestion, asearch query suggestion grouping including: data describing the searchquery suggestion; and data depicting the representative resource set. 2.The method of claim 1, wherein: the resources are images; eachrespective set of resources determined to be responsive to a searchquery suggestion is a respective set of images determined to beresponsive to the search query suggestion; generating the representativeresource set depicting a subset of the respective set of resourcescomprises generating a representative image set depicting a subset ofthe respective set of images determined to be responsive to the searchquery suggestion; each of the search results that references a resourcethat is determined to be responsive to the query comprises an imagesearch result that identifies an image determined to be responsive tothe query; for each search query suggestion grouping: the datadescribing the search query suggestion is textual data describing thesearch query suggestion; and the data depicting the representativeresource set is image data depicting the representative image set. 3.The method of claim 2, wherein generating a representative image setdepicting a subset of the respective set of images comprises, for eachrespective set of images, generating a thumbnail image that depicts therepresentative image set depicting a subset of the respective set ofimages.
 4. The method of claim 2, wherein the search results pageresource includes instructions that cause the user device to: render thesearch query suggestion groupings in a first row on the search resultspage; render the set of image search results in rows adjacent the firstrow on the search results page; and in response to a selection of aparticular search query suggestion grouping: generating a querysuggestion display environment on the search results page between thefirst row and the rows of the image search results; display in the querysuggestion display environment at least the subset of the respective setof images determined to be responsive to the search query suggestion forwhich the search query suggestion grouping is selected; and display atleast the textual data describing the search query suggestion.
 5. Themethod of claim 4, wherein the query suggestion display environmentincludes a user interface element for selecting the search querysuggestion, and upon such selection causes the user device to request asearch results page for image search results that includes image searchresults responsive to the search query suggestion.
 6. The method ofclaim 5, wherein the textual data describing the search query suggestioncomprises text data that is non-duplicative of text of the search query,and wherein the display of the at least textual data describing thesearch query suggestion comprises displaying the text data that isnon-duplicative of text of the search query.
 7. The method of claim 2,wherein the textual data describing the search query suggestioncomprises text data that is non-duplicative of text of the search query.8. The method of claim 2, wherein, for each respective set of images,generating a representative image set depicting a subset of therespective set of images determined to be responsive to the search querysuggestion comprises generating the representative image set only fromthe respective images that are not identified as images determined to beresponsive to the query.
 9. A method performed by a user device, themethod comprising: displaying, in response to a search query for imagecontent, a search results page, the displaying including: displaying aset of image search results that are determined to be responsive to thequery, each image search result including a representative image of animage determined to be responsive to the query; displaying a set ofsearch query suggestion groupings, each search query suggestion groupingincluding: textual data describing a search query suggestion that isbased on the search query; and a representative image set depicting asubset of a respective set of images that are determined to beresponsive to the search query suggestion; in response to a selection ofa particular search query suggestion grouping: generating a querysuggestion display environment on the search results page adjacent tothe particular search query suggestion grouping; displaying in the querysuggestion display environment at least the subset of the respective setof images determined to be responsive to the search query suggestion towhich the search query suggestion grouping corresponds; and display atleast the textual data describing the search query suggestion.
 10. Themethod of claim 9, wherein each representative image set depicting asubset of a respective set of image comprises, for each respective setof images, a thumbnail image that depicts the representative image setdepicting a subset of the respective set of images.
 11. The method ofclaim 9, wherein the textual data describing the search query suggestioncomprises text data that is non-duplicative of text of the search query,and wherein the display of at least textual data describing the searchquery suggestion comprises displaying the text data that isnon-duplicative of text of the search query
 12. A system, comprising: adata processing apparatus; and a memory storage device storinginstructions executable by the data processing apparatus and that uponsuch execution cause the data processing apparatus to perform operationscomprising: receiving, in response to a search query for resourcesreceived from a user device, a set of search query suggestions, eachsearch query suggestion being based on the search query and differentfrom the search query; for each search query suggestion, receiving dataidentifying a respective set of resources determined to be responsive tothe search query suggestion; for each respective set of resources,generating a representative resource set depicting a subset of therespective set of resources; providing to the user device, in responsethe search query, a search results page resource, the search resultspage resource including instructions that cause the user device torender a search results page that includes: a set of search results thatidentify resources that are determined to be responsive to the searchquery; for each search query suggestion, a search query suggestiongrouping including: data describing the search query suggestion; anddata depicting the representative resource set.
 13. The system of claim12, wherein: the resources are images; each respective set of resourcesdetermined to be responsive to a search query suggestion is a respectiveset of images determined to be responsive to the search querysuggestion; generating the representative resource set depicting asubset of the respective set of resources comprises generating arepresentative image set depicting a subset of the respective set ofimages determined to be responsive to the search query suggestion; eachof the search results that references a resource that is determined tobe responsive to the query comprises an image search result thatidentifies an image determined to be responsive to the query; for eachsearch query suggestion grouping: the data describing the search querysuggestion is textual data describing the search query suggestion; andthe data depicting the representative resource set is image datadepicting the representative image set.
 14. The system of claim 13,wherein generating a representative image set depicting a subset of therespective set of images comprises, for each respective set of images,generating a thumbnail image that depicts the representative image setdepicting a subset of the respective set of images.
 15. The system ofclaim 13, wherein the search results page resource includes instructionsthat cause the user device to: render the search query suggestiongroupings in a first row on the search results page; render the set ofimage search results in rows adjacent the first row on the searchresults page; and in response to a selection of a particular searchquery suggestion grouping: generating a query suggestion displayenvironment on the search results page between the first row and therows of the image search results; display in the query suggestiondisplay environment at least the subset of the respective set of imagesdetermined to be responsive to the search query suggestion for which thesearch query suggestion grouping is selected; and display at least thetextual data describing the search query suggestion.
 16. The system ofclaim 15, wherein the query suggestion display environment includes auser interface element for selecting the search query suggestion, andupon such selection causes the user device to request a search resultspage for image search results that includes image search resultsresponsive to the search query suggestion.
 17. The system of claim 16,wherein the textual data describing the search query suggestioncomprises text data that is non-duplicative of text of the search query,and wherein the display of the at least textual data describing thesearch query suggestion comprises displaying the text data that isnon-duplicative of text of the search query.
 18. A memory storage devicestoring instructions executable by a data processing apparatus and thatupon such execution cause the data processing apparatus to performoperations comprising: receiving, in response to a search query forresources received from a user device, a set of search querysuggestions, each search query suggestion being based on the searchquery and different from the search query; for each search querysuggestion, receiving data identifying a respective set of resourcesdetermined to be responsive to the search query suggestion; for eachrespective set of resources, generating a representative resource setdepicting a subset of the respective set of resources; providing to theuser device, in response the search query, a search results pageresource, the search results page resource including instructions thatcause the user device to render a search results page that includes: aset of search results that identify resources that are determined to beresponsive to the search query; for each search query suggestion, asearch query suggestion grouping including: data describing the searchquery suggestion; and data depicting the representative resource set.19. The memory device of claim 18, wherein: the resources are images;each respective set of resources determined to be responsive to a searchquery suggestion is a respective set of images determined to beresponsive to the search query suggestion; generating the representativeresource set depicting a subset of the respective set of resourcescomprises generating a representative image set depicting a subset ofthe respective set of images determined to be responsive to the searchquery suggestion; each of the search results that references a resourcethat is determined to be responsive to the query comprises an imagesearch result that identifies an image determined to be responsive tothe query; for each search query suggestion grouping: the datadescribing the search query suggestion is textual data describing thesearch query suggestion; and the data depicting the representativeresource set is image data depicting the representative image set. 20.The memory device of claim 19, wherein generating a representative imageset depicting a subset of the respective set of images comprises, foreach respective set of images, generating a thumbnail image that depictsthe representative image set depicting a subset of the respective set ofimages.